SMC in the Frequency Domain for Long-Memory Processes

Libraries Importing

Outline

1: Time Series Data ^

1.1 Data Importing ^

I. Monthly Electricity Retail Sales

II. Hourly Electricty Demand (Feb 2021)

III. US MoM CPI (Jan 1980 - May 2022)

IV. R Datasets:

VI. Simulated Time Series:

Plot All Datasets

1.2 Data Pre-Processing and Plotting ^

Pre-processing Pipeline

Plotting Functions

RUN to choose, process, and plot the dataset

2: Time Series Functions ^

2.1 Fourier Frequencies and Periodogram ^

RUN Fourier frequencies and Periodogram

2.2 Spectral Density Function (SDF) ^

$$ \begin{align*} S_Y(f)&=\sigma_\epsilon^2\frac{\prod_{j=1}^q\big|1-\xi_{2j}e^{2\pi i f} \big|^2}{\prod_{j=1}^p\big|1-\xi_{1j}e^{2\pi i f} \big|^2}\frac{1}{\prod_{j=1}^k[4\{\cos(2\pi f)-\lambda_j\}^2]^{\delta_j}} \\ &=\sigma_\epsilon^2\frac{\prod_{j=1}^{Rq}\big|1-\alpha_{2j}e^{2\pi i f} \big|^2 \prod_{j=1}^{Cq}\big|(1-\alpha^*_{2j}e^{2\pi i f - \omega^*_{2j}})(1-\alpha^*_{2j}e^{2\pi i f + \omega^*_{2j}}) \big|^2}{\prod_{j=1}^{Rp}\big|1-\alpha_{1j}e^{2\pi i f} \big|^2 \prod_{j=1}^{Cp}\big|(1-\alpha^*_{1j}e^{2\pi i f - \omega^*_{1j}})(1-\alpha^*_{1j}e^{2\pi i f + \omega^*_{1j}}) \big|^2}\frac{1}{\prod_{j=1}^k[4\{\cos(2\pi f)-\lambda_j\}^2]^{\delta_j}} \end{align*} $$

2.3 Whittle Loglikelihood ^

$$ l(\boldsymbol\eta|Z) = -\sum_{j=1}^J\bigg[\log\{S_Y(f_j)\}+\frac{Z_j}{S_Y(f_j)}\bigg]-\frac{1}{2}\bigg[\frac{Z_0}{S_Y(f_0)}+\frac{Z_J}{S_Y(f_J)}+\log\{Z_0 S_Y(f_0)\}+\log\{Z_J S_Y(f_J)\} \bigg] $$

where $\boldsymbol\eta$ is the vector of parameters $$ \begin{align*} \boldsymbol\eta=(&\alpha_{11},\dots,\alpha_{1R_p},\alpha_{11}^*,\dots,\alpha_{1C_p}^*,\omega_{11},\dots,\omega_{1R_p},\omega_{11}^*,\dots,\omega_{1C_p}^*, \\ &\alpha_{21},\dots,\alpha_{2R_q},\alpha_{21}^*,\dots,\alpha_{2C_q}^*,\omega_{21},\dots,\omega_{2R_q},\omega_{21}^*,\dots,\omega_{2C_q}^*, \\ &\delta_{11},\dots,\delta_{1k},\lambda_{11},\dots,\lambda_{1k},\sigma_\epsilon^2) \end{align*} $$ and $Z=\big(Z_0=\hat{S}_p(f_0)\dots,Z_J=\hat{S}_p(f_J)\big)$ is the vector of periodogram values evaluated at the Fourier frequencies $f_j=\frac{j}{N},\quad j\in{\{0,1,...,J=N/2\}}$.

3: Auxiliary Functions and Testing ^

3.1 Extract Parameters ^

3.2 Plot SDF (with Periodogram and Spectral Peaks) ^

3.3 Test all Functions ^

4: Bayesian Setup ^

4.1 Parameter Bounds ^

4.2 Densities and Sampling Functions ^

Probability Density Functions (PDFs)

Samples from the above PDFs

4.3 Joint Prior, Likelihood, and Posterior ^

5: SMC Sampler ^

5.1 Functions ^

5.1.1 Particle Initialization ^

5.1.2 Bridging Densities ^

Bridging densities $\pi_t(x) \propto \pi(x) L(x|Z)^{\gamma_t}, \: t=0,\dots,N; \: \gamma_0 = 0 <\dots<\gamma_N=1$, where $N$ is the length of the schedule, $\gamma_t$ are the temperatures, $\pi(x)$ is the prior and $L(x|Z)$ is the posterior.

5.1.3 MCMC Proposal ^

Truncated normal proposal for all parameters, except $\sigma_\epsilon^2$, for which the proposal is half-normal, as this parameter has an inverse gamma rather than a uniform prior, and thus only has a lower bound of 0, but no upper bound.

5.1.4 ESS and Resampling ^

5.1.5 SMC Sampler (Master Function) ^

SMC Sampler (Metropolis-Hastings)

SMC Sampler Pro (Gibbs Sampler)

5.2 Post-Processing ^

Helper Functions

5.2.1 Plots and Tables ^

5.2.2 Post-Processing Pipeline ^

5.3 Input and Output (Running the Sampler) ^

5.3.1 Data and Model Selection (Input) ^

5.3.2 Prior Specification and Particle Initialization ^

5.3.3 Running the SMC Sampler (Output) ^

Settings and Hyperparameters

SMC Run

Note: for the sms_sampler_pro function, we need a very small number of N_MCMC_updates, as there are N_params times more MCMC steps than in the en-bloc updating sampler. So, for instance, 3 MCMC steps when there are 10 parameters are equivalent to 30 MCMC steps in the older version.

5.3.4 Post-Processing (Plots and Diagnostics) ^

Import prior runs from file – not fully functional

5.3.5 Observations ^

Notes:

6: GARMA Simulation ^

6.1 AR and MA polynomial reconstruction ^

6.2 ARMA Simulation ^

6.3 General Simulation from Spectrum ^

This is the approximate frequency-domain method in Percival (1992).

Consider the simulated ARMA(3,0,3) data from before. We compute the true spectrum for this data and use it to determine its autocovariance sequence, which we then use to simulate the process using a finite truncation of the white noise process. The methodology is that from Percival (1992).

6.4 GARMA Simulation from Functional Form ^

To be implemented...

6.5 Forecasting ^

To be implemented...

6.6 Saving Some Time Series ^

6.6.1 ARMA(3,3) ^

6.6.2 ARFIMA(0,0.4,0) ^

6.6.3 1-Factor Gegenbauer ^

6.6.4 GARMA(5,1,3) ^

6.6.5 GARMA(3,2,2) ^

7. Other Figures from Article ^

Figure 2.1: ACF plot of ARMA vs ARFIMA vs Gegenbauer

Figure 2.2: AR vs Gegenbauer Peaks

Same GARMA(3,2,2) process as used in section 6.2.1.

Figure 3.1: AR(4) autocorrelation and periodogram

Figure 6.7: Comparison of Perodogram and Tapered Estimator

Figure A.1: Distributions

Figure A.2: Tapering

Table with AR and MA coefs and polynomials for the GARMA(3,2,2) model